Integration Guide Universal Robots

Note: We strongly recommend that you read the Robot communication overview before this integration guide.

Note 2: We strongly recommend that you use the latest version of this integration guide. It is included in the latest version of the Robot module. To download the Robot module, visit the official Photoneo website.

Contents

1 Prerequisites

This guide describes the Robot module for robots running PolyScope 5. Before you start the setup, make sure that your robot controller meets these requirements:

E-Series controller with software version 5.10 or higher
UR-Series controller - the supported software versions will be defined after further testing
CB-Series controller - no longer officially supported
RTDE service enabled on the robot (see 2.2)
Photoneo vision controller with Bin Picking Studio (BPS) 1.12

This version of the robot interface was developed and tested with Universal Robots E-Series v5.22 and v5.11.

Robots with PolyScope X use a separate Robot module. It is described in its own integration guide.

To check the software version of your robot, go to Menu -> About:

PolyScope Getting Started screen with the About menu item

The UR software version is on the first line:

PolyScope About dialog showing the software version

2 Robot controller setup

This chapter describes the network configuration, the transfer of the Robot module files, the TCP setup and the State Server configuration. The robot controller needs these settings to communicate with Bin Picking Studio.

2.1 Network configuration

First, configure the IP address of the robot controller port that communicates with the Photoneo vision controller.

Go to Menu -> Settings:

PolyScope Getting Started screen with the Settings menu item

Open System -> Network. Select Static Address as the network method and enter the IP address and Subnet mask of the robot controller. This manual uses 192.168.1.2 and 255.255.255.0.

Click Apply to confirm the changes.

Note: The subnet mask 255.255.255.0 is the same as a 24-bit subnet mask. For more combinations, see this table: https://dnsmadeeasy.com/support/subnet

Example of a matching network configuration on the vision controller side:

Vision controller IPv4 address: 192.168.1.1 / 24
Robot controller IPv4 address: 192.168.1.2 / 24 (as configured in the previous step)

Robot interface network settings on the vision controller

You can also use the Test Connection button on the Network page of the vision controller to ping the Universal Robots controller.

Depending on the result of the test connection (ping), you get a Robot Available or Robot Unavailable popup message. If the robot is unavailable, check the cabling and the network configuration again.

For more details, see the Network page documentation.

2.2 Enabling Real-Time Data Exchange (RTDE)

The vision controller reads the robot state (joint positions and TCP pose) directly from the UR controller through the Real-Time Data Exchange (RTDE) interface. It needs this data for calibration, hand-eye scanning and robot visualization. Therefore, the RTDE service must be enabled and running on the robot. No Robot State Server program has to run on the robot.

To check that the service is running, go to Settings -> Security -> Services and make sure that RTDE is enabled.

PolyScope Services settings with RTDE enabled

2.3 Loading the Robot module files

The Photoneo UR robot interface consists of 2 script files and several URP template programs:

photoneo_scripts
- photoneo_common.script
- customer_definitions.script
photoneo_examples
- pho_main_basic.urp
- pho_main_basic_hand_eye.urp
- pho_main_basic_multi_vs.urp
- pho_main_change_bbox.urp
- pho_main_change_environment.urp
- pho_main_change_solution.urp
- pho_main_get_object_pose.urp
- pho_main_get_status.urp
- pho_main_multiview_dynamic.urp
- pho_main_multiview_static.urp
- pho_main_reuse_scan.urp
- pho_calibration_basic.urp
- pho_calibration_basic_hand_eye.urp
- pho_calibration_auto.urp

Copy both folders (the URP programs and the scripts) to the robot controller. The URP programs include the scripts through Script nodes (type File). These nodes are in the Photoneo Binpicking Initialization folder (in the calibration programs, at the top of the program). PolyScope keeps a copy of the script inside the URP program. So the copy stored in a URP file can be older than the script files in the module.

Reload the scripts in every program you use. After you copy the files, open the program and select each Script node that includes photoneo_common.script or customer_definitions.script. Open the file again from the copied photoneo_scripts folder and save the program. Check that PHO_ROBOT_ID_STR in the loaded photoneo_common.script is "UNIVERSAL_ROBOTS/v1.12.0".

Note: photoneo_common.script uses the waypoints START_VS_1 … START_VS_8 and END_VS_1 … END_VS_8, and customer_definitions.script calls the subprograms gripper_attach, gripper_detach, gripper_user_1, gripper_user_2 and gripper_user_3. All example programs contain them (the Vision_positions subprogram and the gripper subprograms). Keep them when you build your own program.

You can copy the files to the robot controller in two ways:

USB flash drive
File transfer protocols such as SCP, FTP or SFTP

USB flash drive

Copy the folders from the downloaded module archive to a USB flash drive. Plug the drive into the USB port of the teach pendant or the robot controller.

Click Open -> Program:

Select usbdisk -> Photoneo UR Module -> BPS 1.12 -> photoneo_examples:

PolyScope Load Program dialog with the photoneo_examples folder

For example, to copy pho_main_basic.urp to the robot controller, select the file in the list and click Copy.

PolyScope Load Program dialog with the Copy button

Go to a folder on the robot controller and click Paste to copy pho_main_basic.urp there.

PolyScope Load Program dialog with the Paste button

Repeat these steps for all template URP programs that you plan to use in your project. Copy the photoneo_scripts folder in the same way.

File transfer protocol

You can also copy files to the UR controller remotely, directly from a PC. Use WinSCP, FileZilla or another SFTP/FTP client. Use the following configuration for the connection:

Protocol: SFTP
Host: Robot IP address
Port: 22
User: root
Password: easybot

If you prefer a command line, use the following scp command to copy the files:

$ scp -r photoneo_scripts photoneo_examples root@XXX.XXX.XXX.XXX:/programs/

The default password is “easybot”.

2.4 Tool TCP setup

Warning: The Robot module temporarily overrides the active TCP on every Scan (pho_request_scan) and Scan Meshing (pho_request_trigger_scan) request. It sets the TCP to zero (tool flange), sends the request and then restores the previously active TCP. For Scan, it restores the TCP right after it sends the request. For Scan Meshing, it restores the TCP after the vision controller confirms the request. The vision controller reads the robot pose directly from the UR controller over RTDE, so this makes sure that it gets the flange pose.

Note: Because of this, get_actual_tcp_pose() can return inconsistent values when you call it from parallel threads. Take this temporary TCP override into account in multi-threaded applications.

Bin Picking Studio prefers a zeroed TCP setup. Set all TCP values to zero, because the vision side compensates the TCP offsets internally. Use this configuration also during calibration to ensure correct alignment and accuracy.

2.5 State Server

The UR Robot module has no Robot State Server program. Instead, if the RTDE service on the UR robot is enabled, the vision controller connects to RTDE port 30004. It reads the current joint positions (actual_q) and the Cartesian TCP pose (actual_TCP_pose) from the Universal Robots controller. Nothing has to be started on the robot. The joint positions are used for robot visualization. The TCP pose is essential for calibration and for all hand-eye scan requests.

The robot state from RTDE is also useful for first configuration checks. For example, you can visualize the current robot and gripper state on the Environment page with the Robot Module motion mode, as shown below.

Environment page with the real robot pose

When the Robot State Client is connected to the robot, the robot streams the current joint and tool data to Bin Picking Studio at 10 Hz. The robot pose visualization on the Environment page and the calibration should now work correctly.

To check visually that the robot reports the correct zeroed tool pose, start the calibration and switch from the Texture tab to the Verification tab. Then enable the visualization of Tool Pose (robot controller) in the Axis menu. The frame marker should be centered on the flange, with the Z axis pointing down. It should be identical to the Tool0 (robot model) frame.

Calibration verification with the Tool Pose (robot controller) frame

3 Robot module

The Robot module is designed for easy integration into existing applications.

Note: We strongly recommend that you read Photoneo robotic API and Action requests before this section.

3.1 Connection procedures

Note: This procedure connects to the Action Request Server on the vision controller at the start of the program. You can send requests only after the connection is established. The procedure is defined in photoneo_common.script - do not edit it!

Request	Script definition	Input	Populates
Connect to Action Request Server (vision controller)	`pho_wait_for_server(server_ip, port = PHO_BINPICKING_PORT_DEFAULT, max_attempt = 1)`	`server_ip` - IP address of the vision controller (string) `port` (default 11003) `max_attempt` - number of connection attempts (default 1)	`PHO_ERR_CODE`

After the socket is open, the procedure sends the brand ID UNIVERSAL_ROBOTS/v1.12.0. If the connection cannot be opened, the procedure sets PHO_COM_FAILURE (3), shows a popup and stops the program (see 3.6 Error handling).

3.2 Request list

This section describes the API calls of the Robot module. These procedures are intended for high-level control of the bin picking application. The request ID is the number sent to the vision controller. For a full description of each request, see Action requests.

Note: These procedures are defined in the API section of photoneo_common.script. Do not edit them!

Note: Every procedure first waits until a pending Scan request is confirmed (it calls pho_wait_for_scan_completion() itself). All requests except Scan Regular and Trajectory wait for their own response before they return.

Calibration requests

Request	Script definition	Input	Populates
Calibration Start (ID 25)	`pho_calib_start(solution_id, vision_sys_id)`	`solution_id` `vision_sys_id`	`PHO_ERR_CODE`
Calibration Add Point (ID 5)	`pho_calib_add_point()`	—	`PHO_ERR_CODE`
Calibration Save (ID 27)	`pho_calib_save()`	—	`PHO_ERR_CODE` `pho_calib_acc` - calibration accuracy `pho_camera_pose`
Calibration Stop (ID 26)	`pho_calib_stop()`	—	`PHO_ERR_CODE`

pho_calib_add_point() waits until the robot stands still before it sends the request. The vision controller reads the robot pose over RTDE.

Bin picking requests

Request	Script definition	Input	Populates
Initialize (ID 4)	`pho_request_init(pho_start_pose, pho_end_pose, vision_sys_id = PHO_VISION_ID_DEFAULT, timeout = -1)`	`pho_start_pose` `pho_end_pose` `vision_sys_id` [optional, default 1] `timeout` [optional, default -1 - infinite]	`PHO_ERR_CODE` `pho_start_pose`, `pho_end_pose` (joint positions)
Scan Regular (ID 1)	`pho_request_scan(vision_sys_id = PHO_VISION_ID_DEFAULT)`	`vision_sys_id` [optional, default 1]	`PHO_ERR_CODE` Note: The response is received in the background. Call Wait For Scan to wait for it.
Scan Meshing (ID 28)	`pho_request_trigger_scan(vision_sys_id, tcp_pose = PHO_NO_POSE)`	`vision_sys_id` `tcp_pose` [optional; when not provided, the vision controller uses the pose read over RTDE; the examples pass `get_actual_tool_flange_pose()`]	`PHO_ERR_CODE`
Reuse Scan (ID 29)	`pho_run_loc_on_last_scan(vision_sys_id)`	`vision_sys_id`	`PHO_ERR_CODE` Note: Runs localization again on the last acquired scan without triggering a new scan. Waits for its own response.
Wait For Scan	`pho_wait_for_scan_completion()`	—	`PHO_ERR_CODE`
Trajectory (ID 2)	`pho_request_trajectory(vision_sys_id = PHO_VISION_ID_DEFAULT)`	`vision_sys_id` [optional, default 1]	Note: The response is received by the Trajectory Receive procedure.
Trajectory Receive	`pho_receive_trajectory(timeout = -1)`	—	`PHO_ERR_CODE` `pho_tool_point_invariance` `pho_gripping_point_id` `pho_gripping_point_invariance` `pho_dimension_x` (non CAD) `pho_dimension_y` (non CAD) `pho_dimension_rot` (non CAD) `pho_nn_label` (AI Solutions) `pho_max_z_height` (reserved) `pho_tilt` (reserved)
Get Object Cartesian Pose (ID 8)	`pho_request_object_pose(vision_sys_id)`	`vision_sys_id`	`PHO_ERR_CODE` `pho_object_pose` `pho_dimension_x` (non CAD) `pho_dimension_y` (non CAD) `pho_dimension_rot` (non CAD) `pho_nn_label` (AI Solutions)
Pick Failed (ID 7)	`pho_request_send_pick_failed(vision_sys_id = PHO_VISION_ID_DEFAULT)`	`vision_sys_id` [optional, default 1]	`PHO_ERR_CODE`
Get Vision System Status (ID 21)	`pho_request_get_vs_status(vision_sys_id)`	`vision_sys_id`	`PHO_ERR_CODE` `pho_num_localized` `pho_num_ready` `pho_process_state`
Change Bounding Box (ID 32)	`pho_request_change_bbox(vision_sys_id, bbox_id)`	`vision_sys_id` `bbox_id`	`PHO_ERR_CODE`
Change Environment Scene (ID 15)	`pho_request_change_env_state(state_id)`	`state_id`	`PHO_ERR_CODE`

pho_start_pose and pho_end_pose of the Initialize request can be joint positions (a list of 6 values in radians) or Cartesian poses p[x, y, z, rx, ry, rz]. The examples pass the waypoints START_VS_n and END_VS_n. If the pose matches one of the waypoints START_VS_1 … END_VS_8, the module sends the joint positions of that waypoint. Otherwise, it computes the joint positions with get_inverse_kin(). If a Cartesian pose is unreachable, the module shows a popup and stops the program.

Solution requests

Request	Script definition	Input	Populates
Change Solution (ID 9)	`pho_request_change_solution(solution_id)`	`solution_id`	`PHO_ERR_CODE`
Start Solution (ID 10)	`pho_request_start_solution(solution_id)`	`solution_id`	`PHO_ERR_CODE`
Stop Solution (ID 11)	`pho_request_stop_solution()`	—	`PHO_ERR_CODE`
Get Running Solution (ID 12)	`pho_request_running_solution()`	—	`PHO_ERR_CODE` `pho_running_solution`
Get Available Solutions (ID 13, Deprecated)	`pho_request_available_solutions()`	—	`PHO_ERR_CODE` `pho_available_solutions_count` `pho_available_solutions`

Other requests

Request	Script definition	Input	Populates
Communication Check (ID 100)	`pho_request_comm_check()`	—	`PHO_ERR_CODE`

Auxiliary procedures (also in photoneo_common.script)

Procedure	Description
`pho_enable_debug()`	Sets `PHO_DEBUG` to `True`. The module then writes details about each request and response to the log (`textmsg`).
`pho_info_error(code)`	Returns the error code as text, for example `"NO PART FOUND"`.
`pho_get_start_q(vision_sys_id)` `pho_get_end_q(vision_sys_id)`	Return the joint positions of the waypoint `START_VS_n` / `END_VS_n` for vision system 1 to 8.
`pho_get_start_p(vision_sys_id)` `pho_get_end_p(vision_sys_id)`	Return the Cartesian pose of the waypoint `START_VS_n` / `END_VS_n` for vision system 1 to 8.

3.3 Bin picking procedures

Note: pho_bin_picking() and pho_bin_picking_simple() are defined in customer_definitions.script. Do not edit the pho_bin_picking() function itself. The gripper procedures (gripper_attach, gripper_detach, gripper_user_1 … gripper_user_3) are subprograms of the URP program. They are part of every example program. Edit them for the needs of your application.

Bin picking procedure	Description / Usage
Execute bin picking routine `pho_bin_picking` `(` `pho_start_position = pho_start_pose,` `vision_sys_id = PHO_VISION_ID_DEFAULT` `)`	Description Predefined procedure that executes the received bin picking trajectory. Use it when PATH PLANNING TYPE is PLAN JOINT AND LINEAR TRAJECTORIES. It executes the received operations in order. It executes each trajectory segment as a streamed path, with `servoj` or with `movej` (see 3.4) and calls the matching gripper subprogram for each gripper command. At the end, it stops the robot with `stopj`. Do not edit this procedure directly. To change the execution settings, see 3.4 Bin picking movement parameters. Input parameters: `pho_start_position` - Start pose of the vision system (the examples pass `START_VS_n`). `vision_sys_id` - vision system ID The first waypoint of the trajectory is the Start pose that was sent in the Initialize request. If the robot is not in this pose, `pho_bin_picking()` first moves it there with `movej` and then continues with the picking motion. Warning: If you use multiple Start poses (a different one for each vision system), make sure that the Initialize request was sent with the correct Start pose for the vision system. Keep the path from the current robot position to the Start pose free of obstacles. Note: You can call the procedure only once per received trajectory, and only after `pho_receive_trajectory()` finished without an error. Otherwise, it shows a popup and stops the program.
Execute simple bin picking routine `pho_bin_picking_simple` `(` `pho_start_position = pho_start_pose` `)`	Description Very basic bin picking routine for PATH PLANNING TYPE = PATH PLANNING OFF. It moves the robot with `movej` to the Start pose of the last Initialize request (`pho_start_pose`). Then it executes trajectory 1 (approach), calls `gripper_detach`, executes trajectory 2 (grasp), calls `gripper_attach` and executes trajectories 3 (deapproach) and 4 (end). It does not use the gripper commands configured in Bin Picking Studio.
Gripper attach `gripper_attach` `(` `)`	Description A user-defined subprogram. It typically contains the attach procedure, which runs when the robot grasps the object in the Grasp waypoint. Usage It runs automatically when the robot reaches a waypoint of the grasping method that is configured to execute the Attach procedure.
Gripper detach `gripper_detach` `(` `)`	Description A user-defined subprogram. It typically contains the detach procedure, which runs when the object is placed. The robot operator defines this placing routine. Usage It runs automatically when the robot reaches a waypoint of the grasping method that is configured to execute the Detach procedure. Note: This procedure is usually not configured to run automatically in a waypoint. Call it during placing, which the robot operator implements.
Gripper user-defined 1 `gripper_user_1` `(` `)`	Description A user-defined subprogram.
Gripper user-defined 2 `gripper_user_2` `(` `)`	Description A user-defined subprogram.
Gripper user-defined 3 `gripper_user_3` `(` `)`	Description A user-defined subprogram.

3.4 Bin picking movement parameters

Bin Picking Studio supports up to 10 trajectory segments in one bin picking trajectory. The default number of segments is 4. If needed, you can configure more segments on the Grasping method page of the BPS solution. One trajectory can have up to 300 waypoints in total (PHO_MAX_TRAJ_SIZE).

Depending on the number of joint waypoints in each trajectory segment, the pho_bin_picking() procedure executes the motion in one of these ways:

Streamed execution (default, pho_execution_mode = PHO_EXEC_STREAMING): used when the trajectory segment has three or more points. This is the normal case with path planning enabled. The robot follows a smooth curve through the planned waypoints, with a speed profile limited by pho_stream_vel and pho_stream_acc. A new servoj target is sent in every control cycle. The curve never goes beyond a waypoint, and the speed does not depend on how far apart the planner placed the waypoints.
Old ServoJ execution (pho_execution_mode = PHO_EXEC_SERVOJ): one servoj for each planned waypoint, as up to version 1.12.0. It is kept for compatibility and is controlled by pho_servo_time, pho_lookahead_time and pho_servo_gain.
MoveJ: used when the trajectory segment has only two points (for example, when path planning is off or when linear sampling is too high on short segments). The first point is skipped because the robot is already there.

To change the speed of one segment, change the matching value in the array. For example, to make the “Approach to Grasp” segment slower with streamed execution, decrease the second value in the pho_stream_vel array. To make the same segment slower with MoveJ execution, decrease the second value in the pho_vel array. Where a CNT segment flows into the next segment, the robot keeps moving through the junction and slows down only as much as the change of direction requires.

customer_definitions.script contains three speed parameter sets (SLOW, MEDIUM and FAST) for each execution mode. To select the speed for the trajectories, uncomment the matching group of parameters. By default, the MEDIUM sets are selected. Keep pho_stream_lookahead_time, pho_stream_gain and the PHO_FINE_* tolerances at their delivered values.

#-------------------------------------------------------------------------
#------------------- BIN PICKING SPEED SETTINGS --------------------------
#-------------------------------------------------------------------------

#If the PATH PLANNING TYPE = PLAN JOINT AND LINEAR TRAJECTORIES then choose how planned trajectories are executed
PHO_EXEC_STREAMING = 0   # smooth speed profile, robot target updated every control cycle (recommended)
PHO_EXEC_SERVOJ = 1      # legacy: one servoj per planner waypoint
pho_execution_mode = PHO_EXEC_STREAMING

#Parameters for PHO_EXEC_STREAMING, one value per trajectory (index 0 = approach, 1 = grasp, 2 = deapproach, 3 = end ...)
#Max joint speed [rad/s] and max joint acceleration [rad/s^2] along the planned path. Uncomment one group of parameters
#slow
# pho_stream_vel = [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5]
# pho_stream_acc = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]

#medium speed
pho_stream_vel = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
pho_stream_acc = [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]

#fast speed
# pho_stream_vel = [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]
# pho_stream_acc = [4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0]

#servoj filter settings used while streaming (lookahead 0.03 - 0.2 s, gain 100 - 2000)
pho_stream_lookahead_time = 0.05
pho_stream_gain = 1000

#Parameters for PHO_EXEC_SERVOJ (legacy). Uncomment one group of parameters
#slow
# pho_servo_time = [0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3]
# pho_lookahead_time = [0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2]
# pho_servo_gain = [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000]

#medium speed
pho_servo_time = [0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15]
pho_lookahead_time = [0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15]
pho_servo_gain = [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000]

#fast speed
# pho_servo_time = [0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05]
# pho_lookahead_time = [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
# pho_servo_gain = [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000]

#FINE waypoint settling: robot holds the final target until it is within tolerance and at rest
PHO_FINE_POS_TOL = 0.001     # [rad] max joint error
PHO_FINE_VEL_TOL = 0.01      # [rad/s] max joint speed
PHO_FINE_TIMEOUT = 1.0       # [s] continue anyway (with a log message) after this time

#If the PATH PLANNING TYPE = PATH PLANNING OFF then following parameters control the movements
pho_acc = [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4]
pho_vel = [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]
pho_time = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] # if time is specified (pho_time > 0) pho_acc and pho_vel parameters are ignored
pho_blend = [0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02]

At the end of a segment that ends in a precise (FINE) waypoint (for example, the grasp), the robot holds the last waypoint until it is within PHO_FINE_POS_TOL of it and at rest. Only then is the gripper subprogram called. If this does not happen within PHO_FINE_TIMEOUT, the program continues and logs the remaining joint error. This applies to streamed and to old ServoJ execution. MoveJ segments that end in a FINE waypoint are executed without blending. pho_blend is used only for CNT waypoints.

Warning: Be careful when you enable blending for the MoveJ instruction (PATH PLANNING OFF option). If a waypoint is inside the blending region of the previous or next waypoint, the robot does not execute the movement to this waypoint. This can cause problems for precise waypoints, such as grasping waypoints, where exact positioning is critical. If the blending region of the MoveJ command to the previous waypoint includes the next precise waypoint, the robot skips the movement to this precise waypoint. To avoid this, always keep the blending region smaller than the distance between the waypoints.

3.5 Example programs

The Photoneo UR module contains several URP template programs. They show how to use the requests listed in 3.2 Request list for various use cases. Before you run any of them, reload the scripts as described in 2.3.

URP example	Description
pho_main_basic pho_main_basic_hand_eye	This example shows the basic workflow: how to initialize the system, send a scan request, request a trajectory, receive the trajectory as joint waypoints and execute the motion path. It also shows error handling, because each function returns an error value in `PHO_ERR_CODE`. The hand-eye variant moves to the `Scanning` waypoint before each scan and waits for the scan to finish before it moves on. Note: Make sure that the Start and End waypoints are configured in the `Vision_positions` subprogram, as described in 4.1 Teach positions.
pho_calibration_basic pho_calibration_basic_hand_eye pho_calibration_auto	Calibration examples. See 3.8 Calibration.
pho_main_basic_multi_vs	Same as pho_main_basic, but switches between two vision systems. Both vision systems are initialized at the start. The `VISION_SYS_ID` variable defines which vision system is scanned and asked for trajectory data. The switching is done in a folder at the end of the Photoneo Pick and Place loop folder.
pho_main_change_environment	Same as pho_main_basic, but switches between two environment states. It uses the `pho_request_change_env_state()` request to activate the required environment. The `ENVIRONMENT_ID` variable defines which environment state is activated at the next switch. The vision system is initialized again at the start of each cycle. The switching is done in a folder at the end of the Photoneo Pick and Place loop folder.
pho_main_change_solution	Same as pho_main_basic, but with all requests related to solution switching. At the start, it checks the running solution with `pho_request_running_solution()` and, if needed, stops it and starts solution 1 with `pho_request_start_solution()`. In the loop, it switches between solutions 1 and 2 with `pho_request_change_solution()`. The `SOLUTION_ID` variable defines which solution is activated. The switching is done in the Change solution folder at the end of the Photoneo Pick and Place loop folder.
pho_main_change_bbox	Same as pho_main_basic, but switches between two bounding boxes of the same vision system. The `VISION_SYS_ID` and `BBOX_ID` variables define which bounding box `pho_request_change_bbox()` activates. The switching is done in a folder at the end of the Photoneo Pick and Place loop folder.
pho_main_get_object_pose	Similar to pho_main_basic, but instead of a trajectory, the robot requests the Cartesian pose of the object with `pho_request_object_pose()`. This is useful for applications such as pick verification, slip sheet detection and basic picking tasks. The example computes a `Before_pick` pose 50 mm above the object (`pose_trans` with `p[0,0,-0.05,0,0,0]`) and moves to it before the pick. Note: The system returns the raw Cartesian pose from localization. The object’s STL file defines the origin. No gripping points or invariance transformations are applied. The result is stored in `pho_object_pose`.
pho_main_get_status	Same as pho_main_basic, but with calls of the `pho_request_get_vs_status()` request. You can call this request repeatedly at short intervals during localization. It returns `pho_num_localized` (number of objects localized), `pho_num_ready` (number of objects ready for picking) and `pho_process_state` (1 = vision system still processing, 0 = finished). These variables are often used for advanced decisions, especially to find the best time to start the pick procedure.
pho_main_multiview_static	Static meshing example. It shows how to stitch multiple scans together before localization starts. This is useful for complex scenes or large objects. It uses a static approach: the robot stops at each scanning position to trigger and capture a scan. Requirements: RTDE must be enabled. `MESH_DYNAMIC` is set to `False` to disable dynamic meshing. Procedure: The robot stops at each scanning waypoint (`Start`, `Middle`, `End`) and calls `pho_request_trigger_scan()` followed by `pho_wait_for_scan_completion()`. When capturing is complete, a regular `pho_request_scan()` starts localization (it does not trigger an actual scan). For best performance, keep the total number of scans below 10.
pho_main_multiview_dynamic	Dynamic meshing example. It uses Photoneo Instant Meshing technology together with the Parallel Structured Light technique of MotionCam-3D. Dynamic meshing cannot be used with standard PhoXi 3D Scanners. Procedure: A program thread calls `pho_request_trigger_scan(VISION_SYS_ID, get_actual_tool_flange_pose())` repeatedly while `MESH_DYNAMIC` is `True`, with a pause of `CAPTURE_GAP` seconds between the scans. The main program sets `MESH_DYNAMIC` to `True` at the `Start` waypoint, moves through `Middle` to `End` and sets it back to `False`. A regular `pho_request_scan()` then ends the capturing and starts localization. Increase `CAPTURE_GAP` if the vision controller cannot keep up. Keep the scanning trajectory smooth, avoid abrupt rotations and keep the scanned area within the field of view. Otherwise, tracking can be lost. The total number of scans should not exceed 60.
pho_main_reuse_scan	Same as pho_main_basic_multi_vs, but adds the `pho_run_loc_on_last_scan()` request. This is useful when the scene has not changed since the last scan, but localization must run again with a different configuration (for example, to search for different objects, use different bounding boxes or apply different settings). A regular scan captures the scene for VS1. Then `pho_run_loc_on_last_scan()` reuses the scan data from VS1 for VS2 without a new scan.

pho_main_basic - program structure (comments, the Vision_positions subprogram and the gripper subprograms are shortened):

Photoneo Binpicking Initialization
  Script: photoneo_common.script
  Script: customer_definitions.script
  IP_ADDRESS:="192.168.1.1"
  pho_wait_for_server(IP_ADDRESS)
  VISION_SYS_ID:=1
  pho_request_init(START_VS_1,END_VS_1,VISION_SYS_ID)
  If PHO_OCCURRED_ERR
    Popup: Init request returned an error.
    Halt
  pho_request_scan(VISION_SYS_ID)
  Wait: 3.0
Photoneo Pick and Place loop
  Loop
    pho_wait_for_scan_completion()
    If PHO_OCCURRED_ERR
      Popup: Scan request returned an error.
      Halt
    pho_request_trajectory(VISION_SYS_ID)
    pho_receive_trajectory()
    If PHO_ERR_CODE == PHO_NO_PART_FOUND or PHO_ERR_CODE == PHO_PLANNING_FAILED
      MoveJ
        Scanning
        pho_request_scan(VISION_SYS_ID)
    ElseIf PHO_ERR_CODE != PHO_NO_ERROR
      Popup: Trajectory request returned a fatal or unhandled error code.
    Else
      Picking
        pho_bin_picking(START_VS_1,VISION_SYS_ID)
      Placing
        MoveJ
          Before_Place
        pho_request_scan(VISION_SYS_ID)
        MoveL
          Place
            Call gripper_detach
        MoveL
          Deappr_Place
Vision_positions
  Vision system 1: START_VS_1, END_VS_1
  ...
  Vision system 8: START_VS_8, END_VS_8
gripper_attach
gripper_detach
gripper_user_1
gripper_user_2
gripper_user_3

3.6 Error handling

Each request resets PHO_OCCURRED_ERR to False and PHO_ERR_CODE to 0 when its response arrives. If an error occurs while the requested operation runs, the module sets the global variable PHO_OCCURRED_ERR to True and stores the error code in the global variable PHO_ERR_CODE. We recommend that you add error handling for your application after each synchronous request and response receiving procedure.

The module reacts differently to each kind of error:

Error code from the vision controller (for example, 202 No object found): the module only sets PHO_OCCURRED_ERR and PHO_ERR_CODE. Your program decides what to do.
Wrong bin picking configuration (255): the module shows a popup and stops the program. The details are in the Bin Picking Studio console.
Communication errors (connection failed, wrong response, bad data or checksum, trajectory too long): the module shows a popup and stops the program, because PHO_COM_ERR_STOP is True.
Timeout (5): the module only sets the flag and the error code.
Program flow errors (for example pho_request_trajectory() without pho_receive_trajectory(), or pho_bin_picking() called twice): the module shows a popup and stops the program.

To keep the program running after an error that normally shows a popup and stops the program, set PHO_SKIP_ERR = True in your program. The module then only sets the flag and the error code and clears the socket. Use this only together with your own error handling.

For the error codes with their descriptions and troubleshooting, see Error codes.

The module defines the following error code constants in photoneo_common.script:

Error code	UR Script constant
No error (0)	PHO_NO_ERROR = 0
Service error (1)	PHO_SERVICE_ERR = 1
Unknown request (2)	PHO_UNKNOWN_REQ = 2
Communication error (3)	PHO_COM_FAILURE = 3
Bad data (4)	PHO_BAD_DATA = 4
Timeout (5)	PHO_TIMEOUT = 5
Trajectory too long (6) - set by the module, more than 300 waypoints	PHO_LONG_TRAJECTORY = 6
Unknown error (200)	PHO_UNKNOWN_ERR = 200
Path planning failed (201)	PHO_PLANNING_FAILED = 201
No object found (202)	PHO_NO_PART_FOUND = 202
Vision system not initialized (203)	PHO_NOT_INITIALIZED = 203
Part lost (204)	PHO_PART_LOST = 204
Empty scene (218)	PHO_EMPTY_SCENE = 218
Scene state not found (220)	PHO_SCENE_NOT_AV = 220
Scene state already active (221)	PHO_SCENE_ACTIVE = 221
Wrong bin picking configuration (255)	PHO_WRONG_BP_CONFIG = 255
Invalid solution ID (450)	PHO_INVALID_SOLUTION_ID = 450
Another solution running (451)	PHO_ANOTHER_SOL_RUNNING = 451
Deployment not started (452)	PHO_SOLUTION_START_ERROR = 452
Deployment is not running (453)	PHO_SOLUTION_NOT_RUNNING = 453
Deployment did not stop (454)	PHO_SOLUTION_STOP_ERROR = 454
Solution is not complete (455)	PHO_SOLUTION_INCOMPLETE = 455
Invalid robot brand (457)	PHO_INVALID_BRAND_ID = 457
Solution already running (458)	PHO_SOL_ALREADY_RUNNING = 458
Solution access unauthorized (459)	PHO_SOLUTION_FORBID = 459
Invalid vision system (460)	PHO_INVALID_VS = 460
Calibration running (461)	PHO_CALIB_RUNNING = 461
Calibration not running (462)	PHO_CALIB_NOT_RUNNING = 462
Calibration stop error (463)	PHO_CALIB_STOP_ERROR = 463
Calibration start error (464)	PHO_CALIB_START_ERROR = 464
Calibration disabled (465)	PHO_CALIB_DISABLED = 465
Not enough points (466)	PHO_NOT_ENOUGH_POINTS = 466
Calibration accuracy above the allowed threshold (467)	PHO_ERROR_TOO_HIGH = 467
Scanner not available (468)	PHO_SCANNER_NOT_AVAIL = 468
Calibration not allowed (469)	PHO_CALIB_NOT_ALLOWED = 469

Other codes, for example Invalid bounding box (222), have no constant in the module. Compare PHO_ERR_CODE with the number.

Note: The example programs include basic error handling.

3.7 Info data and variables

Responses can contain additional values (see Info data). The module stores them in these global variables of photoneo_common.script:

Key	Variable	Unit in robot
1	`pho_tool_point_invariance`	index
2	`pho_gripping_point_id`	index
3	`pho_gripping_point_invariance`	index
4	`pho_dimension_x`	mm
5	`pho_dimension_y`	mm
6	`pho_dimension_rot`	degrees (converted from mrad by the module)
7	`pho_nn_label`	integer
8	`pho_max_z_height`	mm (reserved, not sent by Bin Picking Studio yet)
9	`pho_tilt`	degrees (reserved, not sent by Bin Picking Studio yet)
10	`pho_calib_acc`	µm
11	`pho_running_solution`	integer
12	`pho_num_localized`	integer
13	`pho_num_ready`	integer
14	`pho_process_state`	0 / 1

All variables start at -1. They keep the value from the last response that contained them. Poses (pho_object_pose, pho_camera_pose) are UR poses p[x, y, z, rx, ry, rz] in meters and radians (rotation vector).

Other global variables used by the module:

Variable	Meaning
`PHO_OCCURRED_ERR`, `PHO_ERR_CODE`	Error flag and error code of the last request (see 3.6).
`pho_object_pose`	Object pose from Get Object Cartesian Pose.
`pho_camera_pose`	Camera pose from Calibration Save.
`pho_start_pose`, `pho_end_pose`	Joint positions [rad] sent in the last Initialize request.
`pho_available_solutions`, `pho_available_solutions_count`	Result of the deprecated Get Available Solutions request.
`system_status`	The current activity of the module, for example `PHO_WAITING` (0), `PHO_SCANNING` (1), `PHO_PICKING` (200), `PHO_DISCONNECTED` (501). The values are the `PHO_` status constants in photoneo_common.script*.
`PHO_DEBUG`	`True` writes details about each request to the log. Default `False`.
`PHO_SKIP_ERR`	`True` disables the popup and stop on communication errors. Default `False`.
`PHO_COM_ERR_STOP`	Constant, `True`: communication errors show a popup and stop the program.

3.8 Calibration

Requirements: Make sure that RTDE is enabled (see 2.2 Enabling Real-Time Data Exchange (RTDE)). The vision controller reads the robot pose for each calibration point over RTDE. Use a zeroed TCP during calibration (see 2.4). The calibration programs include only photoneo_common.script.

pho_calibration_basic / pho_calibration_basic_hand_eye - first start the calibration manually in Bin Picking Studio. The program connects to the vision controller and moves to Calib_Home. Then it moves to Calib_Pose_1 … Calib_Pose_9 and calls pho_calib_add_point() in each pose. At the end, it returns to Calib_Home. Check the result and save it in Bin Picking Studio.
pho_calibration_auto - automatic recalibration. It requires a successful first calibration and automatic recalibration enabled in the vision system settings. The program runs the whole cycle: it stops the running solution (pho_request_stop_solution()), starts the calibration with pho_calib_start(SOLUTION_ID, VISION_SYS_ID), adds the 9 points, saves the result with pho_calib_save() and ends with pho_calib_stop(). pho_calib_acc and pho_camera_pose contain the result.

Teach all 9 calibration poses before the first run. The calibration error should usually stay below 3 mm. A higher error usually indicates a systematic problem in the calibration setup. If the error is above the allowed threshold, the save request returns error 467.

Important: The calibration object must stay where it was during the first calibration. The calibration ball stays mounted in the same place on the flange or gripper and moves with the robot. The marker pattern stays in the same place in the cell. Otherwise, the automatic recalibration fails.

4 Runtime

When the solution is fully configured on the vision controller, finish the remaining steps on the robot side and run the bin picking program.

4.1 Teach positions

When you open pho_main_basic.urp or another template, you see local poses that you must touch up before you run the program. In pho_main_basic, these are Scanning, Before_Place, Place and Deappr_Place. Other templates use the same idea, but the exact names can differ slightly (for example, Before_place / Deappr_place). Touch up these poses.

PolyScope program tree with waypoints to touch up

You must also define the Start and End positions for all vision systems used in the current solution. These positions are the first and last waypoints of the trajectory. The initialization request of each vision system needs them. Usually you touch them up so that the robot tool is above the center of the bin.

To set the Start and End waypoints, go to the Vision_positions subprogram at the bottom of the program. Select the waypoint START_VS_1 and teach it. Then repeat this for the waypoint END_VS_1.

Note: Do not change the position names. photoneo_common.script uses them directly.

Note: pho_bin_picking() starts the trajectory in the Start pose of the vision system (the first waypoint of the trajectory). So the robot moves there from its current position (see 3.3).

4.2 Gripper commands

The gripper procedures are the subprograms gripper_attach, gripper_detach, gripper_user_1, gripper_user_2 and gripper_user_3 at the end of each example program. They are empty by default (gripper_attach only writes a log message). Configure them to execute the gripper IO commands of your application.

On the Grasping Methods page in Bin Picking Studio, you can configure a gripper command for each major waypoint of the bin picking trajectory. For example, if the Attach procedure is defined at the Grasp waypoint, the pho_bin_picking() procedure calls the gripper_attach() subprogram after the robot reaches the Grasp waypoint.

Grasping method with the Attach procedure at the Grasp stage

4.3 Prerequisites

Do a final check before you run the bin picking interface from the robot side. Make sure that:

The bin picking solution is properly configured on the vision controller
The current scripts are loaded in the program (see 2.3)
The network setup on the robot side is complete and RTDE works (see 2 Robot controller setup)
All vision systems defined in the solution are calibrated
The Start and End poses of all vision systems are touched up
All local poses in the main program are touched up
The gripper procedures are ready and working

4.4 Running the pho_main_basic program

Deploy your solution. If the communication is not established yet, the status of the Action Request Client (Robot) on the Deployment page is DISCONNECTED.

Deployment page with the Action Request Client disconnected

Note: We strongly recommend that you reduce the override speed to 20% before you run the program for the first time.

PolyScope program with the speed slider set to 20%

When the connection is established, the status of the Action Request Client and the Robot State Server changes to CONNECTED. The sensor should now capture the first scan, and localization should start to find objects.

Deployment page with the Action Request Client connected

If there is a pickable object in the scene and the robot controller has received the trajectory for the first object, the robot should start to move towards this object.

If everything looks fine, continue moving the robot towards the first target and check that the path is correct. If the robot stops too far from the object or pushes the object too deep, adjust the settings on the Tool Point or Gripping Point pages in Bin Picking Studio.

If the trajectories look fine, set up your own placing routine and slowly increase the speed back to 100%.

Congratulations, you have successfully deployed the Photoneo Universal Robots interface. You can now improve your application further. Use the CheatSheet and the program templates as a guide.

5 Robot module update

To update your Robot module to a version that is compatible with your Bin Picking Studio version, follow these steps:

Back up your custom settings - make a backup of your customer_definitions.script file, because it contains your custom speed settings
Load the new scripts - copy the new versions of photoneo_common.script and customer_definitions.script to the robot controller. In your main URP program, select the Script nodes that include the old scripts and open the new files instead. Save the program. Your gripper subprograms are part of the URP program and are not affected
Reapply your changes - transfer your changes from the old customer_definitions.script to the new version of the script
Review the API changes - read the API changes in the new version of the Robot module carefully. Update the API calls in your main program where needed

To check the version of photoneo_common.script, look at the script header (flexible_communication v.1.12 library and Robot module version: 1.12.1) and the brand ID PHO_ROBOT_ID_STR. The header of customer_definitions.script says Customer Definitions v.1.12.1. See 6 Version and brand ID.

6 Version and brand ID

Robot module version	1.12.1 (Bin Picking Studio 1.12)
Brand ID (`PHO_ROBOT_ID_STR` in photoneo_common.script)	`UNIVERSAL_ROBOTS/v1.12.0` (24 characters)
Action Request Server port	11003 (`PHO_BINPICKING_PORT_DEFAULT`)
Robot state	Read by the vision controller from RTDE port 30004

The module sends the brand ID right after the connection opens. Do not change it. Robot module 1.12.1 still sends UNIVERSAL_ROBOTS/v1.12.0, because the vision controller rejects brand IDs that it does not know. So 1.12.1 works with every Bin Picking Studio version that supports 1.12.0. The vision controller uses it to select the matching protocol version. For compatible Bin Picking Studio versions, see the Robot module support matrix.

7 Migration

1.12.0 -> 1.12.1: Only the Robot module changed, the communication interface is the same. Planned trajectories are streamed by default, FINE segments end exactly on the last waypoint, and the old ServoJ execution has a lower gain (1000) and longer lookahead times. The speed settings in customer_definitions.script have changed (see 3.4). Replace both scripts as described in 5 Robot module update and copy your speed settings to the new parameters.

1.11 -> 1.12: The Info data protocol has changed. Every value is sent with its own key and is stored in the variables listed in 3.7. New variables: pho_nn_label, pho_max_z_height and pho_tilt. The brand ID is UNIVERSAL_ROBOTS/v1.12.0. The example pho_main_change_enviroment.urp was renamed to pho_main_change_environment.urp. Replace both scripts as described in 5 Robot module update. For details, see BPS migration guide 1.11 -> 1.12.

8 Contact information

Headquarters
Zebra Technologies Slovakia s.r.o.
Plynárenská 6
821 09 Bratislava, Slovakia

Technical support
Contact us at the Help Center.
Visit the Photoneo support pages at www.photoneo.com/support.

Orders and inquiries:
photoneo-sales@zebra.com
photoneo-sales-us@zebra.com